Skip to main content

Serverless Monorepo

Overview​

Raphas serverless monorepo template that uses Lerna and Yarn Workspaces.

  • Designed to scale for larger projects
  • Maintains internal dependencies as packages
  • Uses Gitlabs CI/CD to only deploy services which have been updated
  • Supports publishing dependencies as private NPM packages
  • Uses serverless-bundle to generate optimized Lambda packages
  • Uses Yarn Workspaces to hoist packages to the root node_modules/ directory

This template was created to try solve the following problems:

  • Simplifying Dependencies Management
  • Easier Global Refactoring or Bug Fixing
  • Easier Onboarding of Engineers

Installation​

Install NPM packages for the entire project

$ yarn

How It Works​

The directory structure roughly looks like:

package.json
/constants
/layers
  /transformer
/lib
/packages
  /sample
    index.js
    package.json
/services
  /test
    /src
      index/handler.ts
    package.json
    serverless.yml

This repo is split into 4 directories. Each with a different purpose:

  • packages

    These are internal packages that are used in our services. Each contains a package.json and can be optionally published to NPM. Any changes to a package should only deploy the service that depends on it.

    There is currently a single example package called sample. This has not been added to to NPM.

  • layers

    These are lambda layers. Has a package.json and serverless.yml so they can be deployed seperately. We currently have

    1. transformer. This layer is responsible for transforming xml to json and json to xml for other services.

  • services

    These are Serverless services that are deployed. Has a package.json and serverless.yml. We currently have on example.

    1. test: Includes the sample package.

    More on deployments below.

  • lib

    Any common code that you might not want to maintain as a package. Does NOT have a package.json. Any changes here should redeploy all our services.

The packages/, services/ and layers/ directories are Yarn Workspaces.

Services​

The Serverless services are meant to be managed on their own. Each service is based on our Serverless TS Boilerplate. It uses the serverless-bundle plugin (based on Webpack) to create optimized Lambda packages.

This is good for keeping your Lambda packages small. But it also ensures that you can have Yarn hoist all your NPM packages to the project root. Without Webpack, you'll need to disable hoisting since Serverless Framework does not package the dependencies of a service correctly on its own.

Install an NPM package inside a service.

$ yarn add some-npm-package

Run a function locally.

$ serverless invoke local -f get

Run tests in a service.

$ yarn test

Deploy the service.

$ serverless deploy

Deploy a single function.

$ serverless deploy function -f get

Packages​

Since each package has its own package.json, you can manage it just like you would any other NPM package.

To add a new package:

$ mkdir packages/new-package
$ yarn init

Packages can also be optionally published to NPM.

To use a package:

$ yarn add new-package@1.0.0

Note that packages should be added by specifying the version number declared in their package.json. Otherwise, yarn tries to find the dependency in the registry.

Lib​

If you need to add any other common code in your repo that won't be maintained as a package, add it to the lib/ directory. It does not contain a package.json. This means that you'll need to install any NPM packages as dependencies in the root.

To install an NPM package at the root.

$ yarn add -W some-npm-package

Deployment​

We want to ensure that only the services that have been updated get deployed. This means that, if a change is made to:

  • services

    Only the service that has been changed should be deployed. For ex, if you change any code in get-basket, then post-basket or put-basket should not be deployed.

  • packages

    If a package is changed, then only the service that depends on this package should be deployed. For ex, if sample-package is changed, then get-basket should be deployed.

  • lib

    If any of the lib are changed, then all services will get deployed.

Validators​

For generating of Types validators it's necessary to follow the next steps:

  1. If the types to validate are new and doesn't exist in types project you must generate them adding the corresponding schemas to the corresponding folder in a new branch of the project, to do that you can follow the read me instructions in Schemas Project

  2. Once the merge is done and the corresponding pipelines are executed the new types will appear and be published in the types project Types Project

  3. You can create your validator and use the types and schemas following the below example:

import { inspect } from "util";
import Ajv from "ajv";
import Schema from "@rapharacing/schemas";
import addFormats from "ajv-formats";

import { HybrisProductApi } from "@rapharacing/types";

const product_schema = Schema("hybris/product");
const common_definitions_schema = Schema("hybris/_definitions");

export const ajv = new Ajv({
  allErrors: true,
  strict: true,
  keywords: ["originalRef"],
});

ajv.addSchema(common_definitions_schema, "_definitions.json");
addFormats(ajv);

export type ValidateFunction<T> = ((data: unknown) => data is T) &
  // eslint-disable-next-line @typescript-eslint/ban-ts-comment
  //@ts-ignore
  Pick<Ajv.ValidateFunction, "errors">;
export const isProduct = ajv.compile(
  product_schema
) as ValidateFunction<HybrisProductApi>;

export default function validateProduct(value: unknown): HybrisProductApi {
  if (isProduct(value)) {
    return value;
  } else {
    throw new Error(
      ajv.errorsText(
        // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
        isProduct.errors!.filter((e: any) => e.keyword !== "if"),
        { dataVar: "Product" }
      ) +
        "\n\n" +
        inspect(value)
    );
  }
}

In this example the HybrisProductApi type has been generated using the schemas:

const product_schema = Schema('hybris/product');
const common_definitions_schema = Schema('hybris/_definitions');

In this case It's necessary to load all the relations with other types, contained in the _definitions.json file, if your type is not composed by other types it's not necessary to load the _definitions.json file.

{
  dataVar: 'Product';
}

In this case "Product" references to the original type defined in the schema, and HybrisProductApi, is the name of the type generated in base to that schema.